Formation control for discrete-time multi-agent system with input and output delays under network attacks

This paper addresses the mean square consensus-based formation control for multiple agent system (MAS) with input and output delays under network attacks, process noise, and measurement noise in MAS. Firstly, a time delay-free transformation approach for a multi-agent system with input and output delays is presented. The MAS is transformed into a formal delay-free discrete-time system using the delay-free transformation method. Then a state observer based on Kalman filter is presented to estimate the states of agents. A distributed predictive consensus protocol based on the estimated states of each MAS is proposed for the leader-following MAS. The sufficient and necessary conditions of mean square consensus for MAS under DoS attacks are derived. Finally, a numerical example of consensus-based formation control for multi-agent systems is provided to verify the correctness and effectiveness of the proposed consensus control protocol.

Multi-agent systems can solve practical problems with strong robustness, good reliability, and high efficiency. There exist various multi-agent systems in real life, such as multi-robot systems, multiple satellite systems, air vehicle fleet, autonomous underwater vehicle queue, and so on. With the development of sensor network, distributed computing and network communication, network is more and more adopted for information interaction in multiagent systems. Using network technology, multi-agent systems can carry out long-distance data transmission. Due to the openness of the network protocol, network attacks bring inevitably a series of information security problems 1 . By attacking the communication network among multi-agents, the malicious attacks make the communication information unable to transmit normally, which seriously destroys the stability of distributed multi-agent systems. Therefore, increasing attention has been paid to the security consensus of multiagent systems under network attacks 1,2 . The common ways of network attack are denial of service attacks (DoS) and deception attacks, in which denial of service attacks can make the network resources of multiagent systems exhausted. It can temporarily interrupt the networked system or stop the network service, and it can hinder the normal transmission of data and make the data unable to reach the destination agent. At present, the security consensuses of multiagent systems have made some achievements. For example, considering the denial-of-service attacks, the authors studied the distributed consensus control of multiagent systems by using an input-based event triggering method 2 . For the systems with small time-delay, the effect of time-delay was often ignored. However, for the systems with large time delay or in high performance requirements, the effect of time delay cannot be ignored. To solve the time delays of a cohesive formation, Petrillo and Santini, et al. proposed a novel resilient cooperative adaptive strategy that embeds a mitigation mechanism for cyber-attacks which can cope with different malicious behaviors. Using Lyapunov-Krasovkii approach, the effectiveness of the approach was analytically proven when the leading information cannot be falsified with a high-fidelity vehicular network's emulator PLEXE 3 . Therefore, how to reduce or eliminate the negative effects of time delays has become a hot issue. Wu and He 4 studied the consensus control problem of networked multi-agent systems with communication time-delay under network attacks. A security consensus algorithm for multiagent systems based on intermediate state values was proposed 5 . Considering the different attacking intensity, Zhang and Feng 6 introduced the switched system model to model the denial-of-service attacks and studied the robust consensus control of www.nature.com/scientificreports/ leader-follower multi-agent systems. When the attacking frequency and duration of denial-of-service attacks were less than a certain value, a distributed security control strategy 7 was proposed to achieve the consensus control of multiagent systems. In the actual environment, multi-agent systems are often affected by random noise, so the systems often cannot accurately obtain the real states of other agents. If the measured data with noise is directly used, the consensus control method will be difficult to guarantee the system stability of multiagent systems. Existing studies have made significant contributions to address consensus control with noise problems. However, the paper on the consensus of multi-agent systems with noise under network attacking is not enough. The papers [8][9][10] showed how to realize the mean square consensus control of multi-agent systems with process noise, measurement noise and communication noise. For the multi-agent systems with process noise and measurement noise, the papers 11,12 studied the consensus error analysis on the multiagent systems stability. Xia 13 was concerned with the stochastic stability analysis of networked control systems with random network delay, and then a new control scheme termed networked predictive control was proposed. The control prediction generator was used to provide a set of future control predictions to make the closed-loop system achieve the desired control performance. The Multiagent systems under external disturbances and network imperfections, i.e., communication delay and random packet dropout, have been considered. The delay-free approach is exploited, and this kind of transformation was adopted in Refs. 14,15 . For the discrete-time systems with time-varying state delay, Gao and Chen 15 defined a new Lyapunov functions and made use of novel techniques to achieve delay dependence. The merit of the proposed conditions lies in their less conservativeness. The network-delay compensator was adapted to remove the effects of network transmission delay and data dropout. Liu 16 showed the design and analysis of networked multi-agent predictive control systems via cloud computing, and a cloud predictive control scheme for networked multi-agent systems was proposed to achieve consensus and stability simultaneously and to compensate for network delays actively. The authors concerned the consensus and stability problem of multi-agent control systems via networks with communication delays and data loss, and a networked multi-agent predictive control scheme was proposed to achieve output consensus and compensate for the communication delays and data loss actively 17 . Elahi 18 considered the third-order dynamics with the rate of change of acceleration under the communication delay and random packet dropout. Using the Lyapunov-Krasovskii function, the sufficient conditions were provided. The consensus error dynamics are asymptotically mean-square consensus stable, and a given disturbance attenuation level is achieved in the presence of both network imperfections and disturbances.
A considerable number of research achievements on consensus control with denial-of-service have emerged. To overcome the disadvantages of continuous time sampling, event-triggered mechanism is proposed. Li and Wang 19 addresses the mean square consensus problems for leader-following multi-agent systems under denialof-service attacks, process noise, and measurement noise and a distributed predictive control protocol based on the state estimation information of each agent is proposed for the leader-following multi-agent systems to achieve mean square consensus. Li and Wen 20 proposed a new adaptive distributed resilient control scheme to resist the influence of the DoS attacks, and it can almost ensure the resilient leader tracking under arbitrary DoS attacks with bounded durations and frequencies. In Ref. 21 a Markov model approach is proposed to describe the heterogeneous DoS attacks with a particular form of transition probability. Based on the Markov model, an outputfeedback observer is designed to estimate the missing sensor data. In Ref. 22 the authors design a Dynamic Output Feedback Control (DOFC) algorithm to track the given yaw velocity in presence of event-triggering mechanism and DoS attack. Wang 23 studied the modeling of periodic DoS attacks as Bernoulli distribution, and a novel attack-resilient event-triggered mechanism is put forward to address formation shape problems of the system.
The innovations of this work are as follows. Firstly, considered the time delay in MAS with time delay, a free time-delay transformation for MAS with input and output delays is adopted. Then a states observer based on Kalman filter is presented to estimate the states of multiple agents with random noise. Then a distributed predictive consensus algorithm based on the state estimation information by using Kalman filter and predictive control scheme is proposed, which can effectively achieve the states acquisition of the multi-agent system, so that the leader-follower multi-agent system can achieve better consensus effect, and it further enriches the existing conclusions of prospect theory.

Problem formation
A discrete-time leader-follower MAS with control and output delays is considered. The dynamic model of agent i is described as where k is the discrete sampling time, i ∈ N , agent 0 is the leader, agent i is the follower i , x i (k) ∈ ℜ n i , y i (k) ∈ ℜ l and u i (k) ∈ ℜ m i are the states of i , the output states of agent i and control input of agent i , respectively. A ∈ ℜ n i ×n i , B ∈ ℜ n i ×m i and C ∈ ℜ l×n i is the parameter matrix of the agent i , respectively.h 1 > 0 is the control time delay,h 2 > 0 is the output time delay. w i (k) ∈ ℜ n i and v i (k) ∈ ℜ l is the process noise and measurement noise of the agent i , respectively. Q i (k) and R i (k) is the covariance of w i (k) ∈ ℜ n i and v i (k) ∈ ℜ l , respectively. The initial state x i (0) is uncorrelated with w i (k) and v i (k).
The solution of (1) can be given by (1) www.nature.com/scientificreports/ For the system (Eq. 1), based on the solutions (Eq. 2) a transformation vectors are introduced to convert the time delay system (Eq. 1) into an equivalent delay-free system, the transformed system is described as Meanwhile, a delay-free output transformation vector ỹ i (k) is designed as Based on Eqs. (1), (2), (3) and (4), delay system (Eq. 1) can be rewritten as a delay-free form, which is described as It is assumed that the system (Eq. 5) is uniformly completely controllable and uniformly completely observable.
Denial of Service attack is one of the most common network attacks. DoS destroys the information interaction among agents, and it leads that the agent i neighbor data cannot be transmitted to agent i , even it seriously affects the consensus of multi-agent systems. The assumption of limited DoS attacker's energy is more realistic. For example, many digital devices usually use battery and when they are used as tools for DoS attacks, the power of DoS attackers is limited. In this paper, we assume that the attacker's energy is limited, so limited continuous attacks can be carried out on the MAS and the upper bound of continuous attack time is d k . In addition, the energy constraints of DoS attackers are discussed in Refs. 23,24 . In order to describe the attack probability of DoS, a variable α i (k) subjecting to Bernoulli distribution is defined. When a denial-of-service attack occurs,α i (k) = 1 , otherwise α i (k) = 0 . The probability of occurrence or nonoccurrence of denial-of-service attacks are 25 .
Then the leader-follower multi-agent systems can achieve the mean square consensus.

Main results
Observer design. For the leader-follower multi-agent systems (Eq. 5), due to the influence of process noise and measurement noise, the real values of MAS states variables cannot be obtained. So how to deal with the noised information of MAS states effectively and accurately is very important. Based on Kalman filter, an estimation algorithm is proposed to estimate the states of multiple agents effectively, which can reduce the adverse effects of noise on the MAS. The Kalman filter-based observer of leader-follower multi-agent systems (Eq. 5) is designed as follows: Pr ob{α i (k) = 1} = α Pr ob{α i (k) = 0} = 1 − α.

Controller design.
Due to the adverse effects of the process noise, measurement noise and denial of service attacks, how to achieve the consensus of leader-follower multi-agent systems (Eq. 5) has become a very challenging problem. When the denial-of-service attack occurs, the communication data among agents is blocked, the information interaction process cannot be carried out normally, and the information sent by the leader cannot be received by other followers. In this part, based on Kalman filter theory, the states of multi-agent systems with noise are estimated, so that this control scheme can achieve the accurate and effective acquisition of data and provide a fundamental guarantee for the consensus control of leader-follower multiagent systems. Then a distributed predictive control algorithm based on multi-agent states estimation information is proposed by using predictive control theory to ensure the multi-agent systems achieve the consensus in the maximum sustained attacking time.
Taking agent i for an example. When an attack occurs, the agent i uses the state estimation information of its neighbor agent j before time k to predict the states of agent j from the step k − d k + 1 to step k , where d k > 0 .The algorithm procedure is as follows: where x j (k) is the estimated value of x j (k) at time k,τ is time delay, d k is the upper bound of continuous attack time.
To achieve the consensus control of the leader-follower multi-agent system (Eq. 5), a distributed predictive control algorithm based on the estimated states of multi-agents is designed as follows: where G i is the controller gain,a ij is the weight value on the edge between agent i and agent j and a ij > 0 if agent i receives the data from agent j else a ij = 0.b i =1 represents the agent i receives the data of the leader else b i = 0..

Consensus analysis.
When each agent of the multi-agent systems has process noise and measurement noise, it is difficult to achieve stability for each multi-agent. At the same time, in the process of information interaction among agents of this system, if the system is attacked by denial of service, which makes it more difficult to control the multi-agent system.

Theorem 1
Under the denial-of-service attack, for the multi-agent system (Eq. 5) with process noise and measurement noise, if matrix A x and matrix A − K i (k)CA satisfy Schur stability theorem, then the multi-agent system (Eq. 5) can achieve mean square consensus.
In Theorem 1,A x is described as (6) x i (k + 1|k + 1) =x i (k + 1|k ) + K i (k + 1)(ŷ i (k + 1) −ŷ i (k + 1|k )), www.nature.com/scientificreports/ Proof For agent i , we define the error variable e i (k + 1) as Combining (2) and (3), we can obtain The average value of e i (k + 1) is described as Equation (7) can be obtained by iterative calculation Using the iterative calculation for Eq. (5), we can obtain Then Eq. (14) can be rewritten as Combining Eqs. (11), (13) and (15), for agent j , we can obtain its equation www.nature.com/scientificreports/ In the same way, for agent 0 , we can obtain its equation Combining Eqs. (16) and (17), the proposed control algorithm (9) can be rewritten as Combining Eqs. (5) and (18), we can obtain For agent i , we define error variables  www.nature.com/scientificreports/ Obviously, when the error Eq. (24) is asymptotically stable, the leader-follower multiagent system (Eq. 5) can achieve mean square consensus control. According to the Schur stability theorem, if and only if the matrix A x and A − K i (k)CA satisfy Schur stability theorem, there are Furthermore, when the leader-follower multi-agent system (Eq. 5) can achieve mean square consensus, and then the system (Eq. 1) can achieve the mean square consensus.

Simulation analysis
Consider a multi-agent system composed of one leader and six followers. The MAS' topology is shown in Fig. 1, in which agent 0 is the leader, the others are followers, and agent 1 and agent 4 can receive the leader's information.

Example 1
The probability of denial-of-service attack α = 0.6 . The upper bound of continuous attack time d k = 2 and the initial states of all agents are.
The leader's control input u 0 (k) = 0.8. Choose a set of system gain The control time delay h 1 = 2 , and the output time delay h 2 = 1.Note that the matrices A x and A − K i (k)CA(∀i ∈ N) matrices are of Schur stability. Figure 2 shows the consensus control motion simulation   Figure 5 shows the motion simulation of the consensus control of agents at constant velocity and heading angle, Fig. 6 shows the velocity consensus control and Fig. 7 Figure 8 shows the motion simulation of the consensus control of agents at constant velocity and heading angle with one second interval attacks. Figure 9 shows the velocity consensus control and Fig. 10 shows the heading angle consensus control. Although the existing interval attacks, the seven agents with estimators can reach consensus on velocity and heading. All the seven agents can realize the real-time consensus acquisition of information. Figure 11 shows the motion simulation of the consensus control of seven agents at variable velocity and heading angle with attacks. Figure 12 shows the velocity consensus control and Fig. 13         www.nature.com/scientificreports/